Investigating user preferences in utilizing a 2D paper or 3D sketch based interface for creating 3D virtual models

Computer modelling of 2D drawings is becoming increasingly popular in modern design as can be witnessed in the shift of modern computer modelling applications from software requiring specialised training to ones targeted for the general consumer market. Despite this, traditional sketching is still prevalent in design, particularly so in the early design stages. Thus, research trends in computer-aided modelling focus on the the development of sketch based interfaces that are as natural as possible. In this report, we present a hybrid sketch based interface which allows the user to make draw sketches using offline as well as online sketching modalities, displaying the 3D models in an immersive setup, thus linking the object interaction possible through immersive modelling to the flexibility allowed by paper-based sketching. The interface was evaluated in a user study which shows that such a hybrid system can be considered as having pragmatic and hedonic value.peer-reviewe

Automated page turner for musicians

An increasing number of musicians are opting to use tablet devices instead of traditional print media for their music sheets since the digital medium offers the benefit of storing a lot of music in a compact space. The limited screen size of the tablet devices makes the music difficult to read and musicians often opt to display part of the music page at a time. With fewer music lines on display, the musician will then have to resort to scrolling through the music to read the entire score. This scrolling is annoying since the musicians will need to remove their hands from the instrument to interact with the tablet, causing a break in the music if this is not done quickly enough, or if the tablet is not sufficiently responsive. In this paper, we describe an alternative page turning system which automates the page turning event of the musician. By actively monitoring the musician's on-screen point of regard, the system retains the musician in the loop and thus, the page turns are attuned to the musician's position on the score. By analysing the way the musician's gaze changes between attention to the score and the instrument as well as the way musicians fixate on different parts of the score, we note that musicians often look away from the score and toward their hands, or elsewhere, when playing the instrument. As a result, the eye regions fall outside the field-of-view of the eye-gaze tracker, giving rise to erratic page-turns. To counteract this problem, we create a gaze prediction model that uses Kalman filtering to predict where the musician would be looking on the score. We evaluate our hands-free page turning system using 15 different piano songs containing different levels of difficulty, various repeats, and which also required playing in different registers on the piano, thus, evaluating the applicability of the page-turner under different conditions. Performance of the page-turner was quantified through the number of correct page turns, the number of delayed page turns, and the number of mistaken page turns. Of the 289 page turns involved in the experiment, 98.3% were successfully executed, 1.7% were delayed, while no mistaken page turns were observed.peer-reviewe

Exploiting artistic cues to obtain line labels for free-hand sketches

Artistic cues help designers to communicate design intent in sketches. In this paper, we show how these artistic cues may be used to obtain a line labelling interpretation of freehand sketches, using a cue-based genetic algorithm to obtain a labelling solution that matches design intent. In the paper, we show how this can be achieved from off-line or paper based sketches, thereby allowing designers greater flexibility in the choice of sketching medium.peer-reviewe

Vectorisation of sketched drawings using co-occurring sample circles

This paper presents a drawing vectorisation algorithm which uses multiple concentric families of circles placed in a dense grid on the image space. We show that any off-centered junction within the family of circles can be located and hence show how these junction points may be linked to neighbouring junction points, thereby creating a vector representation of the drawing geometry. The proposed algorithm identified 98%98% of the junctions in the drawings on which it was evaluated, each within a localisation error of 4.7±2.34.7±2.3 pixels, resulting in straight line vectors which are well placed with respect to the drawn edges.peer-reviewe

An evolutionary approach to determining hidden lines from a natural sketch

This paper focuses on the identification of hidden lines and junctions from natural sketches of drawings that exhibit an extended-trihedral geometry. Identification of hidden lines and junctions is essential in the creation of a complete 3D model of the sketched object, allowing the interpretation algorithms to infer what the unsketched back of the object should look like. This approach first labels the sketched visible edges of the object with a geometric edge label, obtaining a labelled junction at each of the visible junctions of the object. Using a dictionary of junctions with visible and hidden edges, these labelled visible junctions are then used to deduce the edge interpretation and orientation of some of the hidden edges. A genetic algorithm is used to combine these hidden edges into hidden junctions, evolving the representation of the hidden edges and junctions until a feasible hidden view representation of the object is obtained.peer-reviewe

Genetic algorithm for line labeling of diagrams having drawing cues

Drawings are an integral part of the design process, helping designers communicate abstract concepts to others. In this paper we propose a genetic algorithm that successfully exploits cues present in drawings in a line labeling algorithm for sketches.peer-reviewe

Scribble vectorization using concentric sampling circles

In this paper we introduce a path extraction algorithm for multi-stroke scribbled paths by making use of path-centred concentric sampling circles. Circle and line geometry is then exploited to efficiently obtain piece-wise linear models of the multi-stroke segments in the drawing. Parzen-window estimation is used to obtain the probability distribution of the grey-level profile of the sampling circles to determine the intersecting angle of the sampling circle with the stroke segments and hence determine the line model parameters. The results obtained show that the algorithm identifies the line models accurately while reducing considerably the computational time required to obtain the line models.peer-reviewe

Investigating user response to a hybrid sketch based interface for creating 3D virtual models in an immersive environment

This research was done in collaboration with the Fraunhofer Institute for Production Systems and Design Technology Berlin. It was supported by VISIONAIR, a project funded by the European Commission under grant agreement 262044.Computer modelling of 2D drawings is becoming increasingly popular in modern design as can be witnessed in the shift of modern computer modelling applications from software requiring specialised training to ones targeted for the general consumer market. Despite this, traditional sketching is still prevalent in design, particularly so in the early design stages. Thus, research trends in computer-aided modelling focus on the the development of sketch based interfaces that are as natural as possible. In this paper, we present a hybrid sketch based interface which allows the user to make draw sketches using offline as well as online sketching modalities, displaying the 3D models in an immersive setup, thus linking the object interaction possible through immersive modelling to the flexibility allowed by paper-based sketching. The interface was evaluated in a user study which shows that such a hybrid system can be considered as having pragmatic and hedonic value.peer-reviewe

Sketch-based interaction and modeling: where do we stand?

Sketching is a natural and intuitive communication tool used for expressing concepts or ideas which are difficult to communicate through text or speech alone. Sketching is therefore used for a variety of purposes, from the expression of ideas on two-dimensional (2D) physical media, to object creation, manipulation, or deformation in three-dimensional (3D) immersive environments. This variety in sketching activities brings about a range of technologies which, while having similar scope, namely that of recording and interpreting the sketch gesture to effect some interaction, adopt different interpretation approaches according to the environment in which the sketch is drawn. In fields such as product design, sketches are drawn at various stages of the design process, and therefore, designers would benefit from sketch interpretation technologies which support these differing interactions. However, research typically focuses on one aspect of sketch interpretation and modeling such that literature on available technologies is fragmented and dispersed. In this paper, we bring together the relevant literature describing technologies which can support the product design industry, namely technologies which support the interpretation of sketches drawn on 2D media, sketch-based search interactions, as well as sketch gestures drawn in 3D media. This paper, therefore, gives a holistic view of the algorithmic support that can be provided in the design process. In so doing, we highlight the research gaps and future research directions required to provide full sketch-based interaction support